Papers with sample complexity
How well do you know your summarization datasets? (2021.findings-acl)
Copied to clipboard
| Challenge: | State-of-the-art summarization systems are trained on massive datasets scraped from the web. |
| Approach: | They manually analyse 600 samples from three popular summarization datasets . they use a six-class typology which captures different noise types and degrees of summarizing difficulty. |
| Outcome: | The proposed model performs better on large datasets than on the current models. |
A Two-Stage Approach towards Generalization in Knowledge Base Question Answering (2022.findings-emnlp)
Copied to clipboard
Srinivas Ravishankar, Dung Thai, Ibrahim Abdelaziz, Nandana Mihindukulasooriya, Tahira Naseem, Pavan Kapanipathi, Gaetano Rossiello, Achille Fokoue
| Challenge: | Existing approaches for Knowledge Base Question Answering focus on a specific knowledge base or evaluating it on underlying knowledge base requires non-trivial changes. |
| Approach: | They propose a framework that separates semantic parsing from knowledge base interaction . they propose KBQA framework that allows generalization across knowledge bases . |
| Outcome: | The proposed framework achieves comparable or state-of-the-art performance on datasets with a different knowledge base. |
APRIL: Interactively Learning to Summarise by Combining Active Preference Learning and Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | Existing preference-based interactive learning methods suffer from high sample complexity, i.e. they need to interact with the oracle for many rounds in order to converge. |
| Approach: | They propose a method which learns from users’ preferences instead of reference summaries by interacting with the oracle for multiple rounds and leveraging active learning, preference learning and reinforcement learning techniques. |
| Outcome: | The proposed method significantly advances the state of the art in both simulation and real-user experiments. |
Generative or Discriminative? Revisiting Text Classification in the Era of Transformers (2025.emnlp-main)
Copied to clipboard
Siva Rajesh Kasa, Karan Gupta, Sumegh Roychowdhury, Ashutosh Kumar, Yaswanth Biruduraju, Santhosh Kumar Kasa, Pattisapu Nikhil Priyatam, Arindam Bhattacharya, Shailendra Agarwal, Vijay Huddar
| Challenge: | generative classifiers exhibit lower sample complexity but higher asymptotic error in simple linear settings, a trade-off that remains unexplored in the transformer era. |
| Approach: | They propose to evaluate generative and discriminative architectures for text classification using a generative model that learns the conditional probability distribution P (y|x) generative models are known to work better in low-data settings, giving rise to the classical 'two regimes' phenomenon for classification. |
| Outcome: | The proposed models show that the classical 'two regimes' manifests distinctly across different architectures and training paradigms. |
Learning from Evolving Training Dynamics: An Entropy-Maximizing Data Curation Strategy for LLM Supervised Post-Training (2026.acl-long)
Copied to clipboard
| Challenge: | EVO-Curate is a dynamic data curation framework that synchronizes sample complexity with the maturing capacity of the Large Language Model (LLM). |
| Approach: | They propose a dynamic data curation framework that synchronizes sample complexity with maturing capacity of the Large Language Model (LLM) they use an Adaptive Dynamics Measurer to synthesize instantaneous difficulty and historical variability into a multidimensional utility score. |
| Outcome: | The proposed framework outperforms standard training baselines and traditional CL methods on instruction following, mathematical reasoning, and code generation architectures while maintaining manageable computational overhead. |
Exploring the Learning Capabilities of Language Models using LEVERWORLDS (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models of stochastic learning involve learning general structure rules and specific properties of the instance. |
| Approach: | They propose a framework that allows the generation of physics-inspired worlds that follow a similar generative process with different distributions and their instances can be expressed in natural language. |
| Outcome: | The proposed framework allows the generation of physics-inspired worlds that follow a similar generative process with different distributions and their instances can be expressed in natural language. |